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Abstract 

We study the asymptotic behavior of multiscale stochastic gene networks using 
weak limits of Markov jump processes. Depending on the time and concentration scales 
of the system we distinguish four types of limits: continuous piecewise deterministic 
processes (PDP) with switching, PDP with jumps in the continuous variables, averaged 
PDP, and PDP with singular switching. We justify rigorously the convergence for the 
four types of limits. The convergence results can be used to simplify the stochastic 
dynamics of gene network models arising in molecular biology. 

MSG: 60J25, 60J75, 92B05. 
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test functions. 



X 



1 Introduction 

Modern molecular biology emphasizes the important role of the gene regulatory net- 
works in the functioning of living organisms. Recent experimental advances in molec- 
ular biology show that many gene prod ucts do not follow determ i nistic dynamics and 

should be modeled as random variables (jKepler and ElstonI (|200l[ ) maufmann and van Oudenaarden 
(|2007l) 1. 

Markov processes approac hes to gene network s dynamics, originating from the 



pioneering ideas of Delbriick (| M. Delbriick I (|l940h ). capture div erse featur e s of t he 
experimentally observed expression variability, such as bursting ( Cai et all (2006)), 



variou s types of steady-state distributions of RNA and protein num bers (iKaern et a,L. 
( 2005)), noise amplification or reduction by n etwork propagation ( PaulssonT i 2004[) " 



Warren et al. ( 2006j )), clock de-synchronization jBarkai and Leiblen (20001) ). stochastic 
transitions in cellular memory storage circuits (jKaufmann et al.l ( 2007h ). 

However, the study of the full Markov dynamics of biochemical networks is a dif- 
ficult task. Even the simplest Markovian model, such as the production module of a 
single protein involves tens of variab l es and biochemical reaction s and an equivalent 
number of parameters ( Kierzek et al. (2001); Krishna et al.l ( 20051 )). The direct simu - 
lation of such models by the Stochastic Simulation Algorithm (SSA) ( Gillespie ( 1976[ )) 
is extremely time consuming. 



1 



In order to increase computational efficiency, several accelerated simula tion algo- 

rithms are hybrid and treat f a st biochemical reacti ons as continuous variables ( Haseltine and Rawlingsl 
(|2002h : lAlfonsi et all (|2005h : lAlfonsi et all (|2004l )). Similar approaches re ducing; fast 



reacti ons can be justified by diffusion approximations for Markov processes (jBall et al 



A different hybrid approach is to distinguish between molecular species according 
to their abundances. Species in small amounts can be treated as discrete variables, 
whereas species in large amounts can be considered continuous. It has been proposed 
that, the dynamics of gene networks with well separated abundances, can be we ll 
approximate d by p iecewise deterministic Markov processes (jRadulescu et al.l (|2007l ): 



Cruduetal 



(I2OO9I )). Piecewise deterministic processes (PDP) are used in opera- 
tional research in re l ation with optimal contro l and various technological applications 
Boxma et all (|2005l ): [Ghosh and Bagchil (|2005h : |Pola et all (|2003f) : [Buiorianu and LvgerosI 



2004h ). Their popularity in physical, chemical and biological sciences is also steadily 



increasing as they provide a natura l fram e work to deal with i ntermittent phenomena 
in many contexts ( Radulescu et al.l ( 2007 ): Zeiser et al. ( 2008^ ). 

By looking for the best PDP approximation of a stochastic network of biochem- 
ical reactions, and depen ding on the t ime s cales of the reaction mechanism, we can 
distinguish several cases (jCrudu et al. I (l2009l) ): 



• Continuous PDP with switching: continuous variables evolve according to or- 
dinary differential equations. The trajectories of the continuous variables are 
continuous, but the differential equations depend on one or several discrete vari- 
ables. 

• PDP with jumps in the continuous variables: the same as the previous case, but 
the continuous variables can jump as well as the discrete variables. 

• Averaged PDP: some discrete variables have rapid transitions and can be aver- 
aged. The resulting approximation is an averaged PDP. 

• Discontinuous PDP with singular switching: the continuous variable has two 
time scaling. The switch between the two regimes is commanded by a discrete 
variable. The rapid parts of the trajectory of the continuous variable can be 
approximated by discontinuities. 

In this paper, we justify rigorousl y these approximati ons that were illustrated by 
models of stochastic gene expression in lCrudu et al. ( 2009f ). More precisely, we present 
several theorems on the weak convergence of biochemical reactions processes towards 
piecewise deterministic processes of the type specified above. The resulting piecewise 
deterministic processes can be used for more efficient simulation algorithms, also, in 
certain cases, can lead to analytic results for the stochastic behavior of gene networks. 
Higher order approximations of multiscale stochastic chemical kinetics, corresponding 
to stochastic differential equations with jumps, though not discussed in this paper, 
represent straightforward extensions of our results. 

The structure of this article is as follows. In section 2, we present the PDP, a useful 
theorem on the uniqueness of the solution of a martingale problem and the Markov 
jump model for stochastic regulatory networks. The four remaining sections discuss 
the asymptotic behaviors of the models, corresponding to the four cases presented 
above. 



2 Piecewise Deterministic Processes 

We begin with a brief description of Piecewise Deterministic Processes (PDP) and 
collect useful result s on these. W e do not consider PDPs in their full generally The 
reader is refered to David (1993) for further results. 



Standard conditions: 
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In this article, a PDP taking values in £■ = M" x N'' is a process xt = {yt.vt), 
determined by its three local characteristics : 

1. For all I' G N"*, a Lipschitz continuous vector field in R", denoted by Fy^ which 
determines a unique global flow (t)y{t,y) in M" such that, for t > 0, 

We also use the notation: F{y, u) = Fy{y). 

2. A jump rate X : E ^ R+ such that, for each a; = {y, v) g E, there exists e{x) > 
such that 

\{4>v{t, y),'^)dt < OO. 



3. A transition measure Q : E ^ x i— )■ Q{-]x), where 'P{E) denotes the set 

of probability measures on E. We assume that Q{{x}\ x) = for each x € E. 

From these standard conditions, a right-continuous sample path {xt : t > 0} starting 
at a; = {y, v) £ E may be constructed as follows. Define 

Xt{uj):=(i3y{t,y), forO<t<ri(w), 

where Ti (w) is the realization of the first jump time Ti , with the following distribution : 



> t) = cxp ( - ^ KMs. y),i')ds^ =■■ H(t, x), 



t e 



We have then x _ (w) = {(/)i,{Ti{uj),y),h'), and the post-jump state XTi{u:)i'^) has 

^1 <'^) 

the distribution given by : 

Px(a^Ti e A\Ti =t) = Q{A- {Mt, y).^)) 

on the Borel sets A of E. 

We then restart the process at xt-i(ui){'^) a-nd proceed recursively according to 
the same procedure to obtain a sequence of jump-time realizations Tl{LJ)^T2{uJ)^ ■ ■ ■■ 
Between each of two consecutive jumps, xt{Lu) follows a deterministic motion, given 
by the flow corresponding to the vector field F. 

Such a process xt is called a PDP. The number of jumps that occurr between the 
times and t is denoted by 

k 

It can be shown that xt is a stron g Markov process with right-continuous, left- 
limited sample paths (see David ( 19931 )). The generator A of the process is formally 
given by 

Af{x) ^ F,{x) ■ Vyfix) + X{x) f (/(z) - f{x))Q{dz- x) (1) 

J E 

for each x — (y, v) G E, we have denoted by Vy the gra dient with re spect to the 
variable y G M" . The domain of A is described precisely in iDavis ( 1993h . We do not 
need such a precise description and just note that A is well defined for / G 5, the set 
of functions / : i? — >■ M such that : 

El. / is bounded, 

E2. for aU v G N^, f{-,v) G C^{W^), 

E3. its derivatives are bounded uniformly in E. 
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For / G we denote by 



Lf = snp \\Dyf\\^ = sup \\Dyf{y,v)\\, (2) 



the Lipschitz constant of / with respect to the variable y. 

The space 5 is a Banach space when endowed with the norm 

+ Lj. 



(3) 



If Z is a Banach space, Bb{Z) is the set of bounded Borel measurable functions 
on Z ; C^{Z) is the set of C'^-differentiable functions on Z , such that the derivatives, 
imtil the /c-th order, are bounded ; Ch{Z) is the set of bounded continuous functions 
on Z . Also D(R+; Z) is the set of process defined on R+ with right-continuous, left- 
limited sample paths defined on R+ and taking values in Z and C(R+; Z) is the set of 
continuous process defined on R+ and taking values in Z . 

The PDPs considered in this paper will always satisfy the following property : 

Hypothesis 2.1 The three local characteristics of the PDF satisfy the standard con- 
ditions given above. The jump rate A is -differentiahle with respect to the variable 
y G R". For every starting point x ~ {y^v) ^ E and t G M"^, we suppose E(A't) < oo. 

Remark 2.2 E(iVt) < oo implies in particular that Tk(uj) — >■ oo almost surely. This 
assumption is usually quite easy to check in applications, but it is hard to formulate 
general conditions under which it holds, because of the complicated interaction between 
F, X and Q. It can be shown for instance that if X is bounded, then E(iVt) < oo (cf. 
iDavi^ h99A )). 

For some results, we need the following stronger property : 

Hypothesis 2.3 The functions F, X and x n- X{x) f{z)Q{dz;x), with f E £, 
are bounded on E, -difjerentiable with respect to the variable y G R" and their 
derivatives with respect to y are also bounded. 



When Hvpothesis 12.31 is satisfied, we set 

A/f = \\F\\oo, Lf = sup \\DyF\\^, 

M\ = ||A||oo, L\ = sup llDj/Alloo- 
and Lq a constant such that, for all / G £ and x — {y^v) ^ E : 



< Lc 



Dy X{x) / f{z)Q[dz-x) 



E 



Denote by (Pt)t>o the transition semigroup associated to the PDF constructed above 
and by the law of the PDF starting from x E E. Then, P^ is a solution of the 
martingale problem associated to A in the following sense: 

f{xt)-f{x)- f Af{xs)ds 
Jo 



is a local martingale for any / G f (see David ( 19931 )). Moreover, if Hypothesis 



holds, it is a bounded martingale. As usual, we have denoted by {xt)t>o the canonical 
process on Z)(R+; i?). 

The following results gives a uniqueness property for this martingale problem. It 
will enable us to characterize the asymptotic behavior of our stochastic regulatory 
networks. 
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Theorem 2.4 If Hypothesis \2.^ is satisfied, then the law of the PDF determined by F, 
X, and Q is the unique solution of the martingale problem associated to the generator 
A. 

The proof of the theorem is given in the appendix. 



Markov jump model for stochastic regulatory networks: known results 

We consider a set of chemical reactions Rr, r ^ TZ;TZis supposed to be finite. These 
reactions involve species indexed by a set S ~ 1, • • • , M, the number of molecules of 
the specie i is denoted by and X G N*^ is the vector consisting of the n^'s. Each 
reaction i?,. has a rate Xr{X) which depends on the state of the system, described by 
X and corresponds to a change X X + jr, Ir . 

Mathematically, this evolution can be described by the following Markov jump 
process. It is based on a sequence {Tk)k>i of random waiting times with exponential 
distribution. Setting Tq = 0, = ti + ■ • ■ + t.;, X is constant on [Ti_i, Ti) and has a 
jump at Fi. The parameter of is given by X^reK '^'"("'^(-^'-i))- 



in >t)=exp(-J2 A,.(^(T,_i))t 



ren 



At time Ti, a reaction ?■ GlZis chosen with probability Ar(X(T!i_i))/ ^^.^^ Ar(X(T!i_i)) 
and the state changes according to X — > X + 7^.: 



This Markov process has the following generator (see lEthier and Kurtz) (|198' 



We do not need a precise description of the domain of A, the above definition holds 
for instance for functions in Cb(M*^). 

In the applications we have in mind, the numbers of molecules have different scales. 
Some of the molecules are in small numbers and some are in large numbers. Accord- 
ingly, we split the set of species into two sets C and D with cardinals Mc and Md. 
This induces the decomposition X = {Xc,Xd), 7^ = (7^,7^^)- For i ^ D, Ui is of 
order 1 while for i £ C, n.i is proportional to N where is a large number. For i C, 

setting hi = Ui/N, hi is of order 1. We define xc = J^-^c and x = (xcXd). 

We also decompose the set of reactions according to the species involved. We set 
TZ = TZd U TZc U TZdc- A reaction in TZd (resp. TZc) produces or consumes only 
species in D (resp. C). Also, the rate of a reaction in TZn (resp. TZc) depends only 
on Xd (resp. xc)- A reaction in TZdc has a rate depending on both xc and Xd and 
produces or consumes, among others, species from C or D. 

The rate of a reaction in r G TZc is also large and of order N and we set \r — j^. 
In general, reactions in TZd or TZdc have a rate of order 1. 

Introducing the new scaled variables, the generator has the form: 



AfixcXo) - 



E 

reUc 



f{xc 



1 

N 



1?.Xd) 



KxcXd) 



N\r{xc) 



E 



/(^c + ^7.'', Xd + 7,'') - f{xc, Xd) 



XrixcXo) 



[f{xc,XD+l^)~ f{xc,XD)]Xr{XD). 



relZr 



Assuming that the scaled rates are with respect to xc, it is not difficult to see 
that a N ^ (X, the two sets of species decouple. Indeed, reactions in TZdc do not 
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happen sufficiently often and they do not change xc in a sufficiently large manner. 
The limit would simply give a set of differential equations for the continuous variable 
Xc, which evolves without influence of X^. The discrete var iable would h ave i ts own 
dynam ic made of jumps. These results have been shown by Kurtz ( 1971 ) and Kurtz! 
(Il978l ). 

In the following sections, we consider more general systems where other types of 
reactions may happen and which yield different limiting systems 



3 Continuous piecewise deterministic process 



In this section, we assume that some of the reactions in a subset Si of TZdc a-re such 
that their rate is large and scales with TV. We again set Xr = jjXr for r ^ Si. We 
assume that these equations do not change Xjj, in other words 



7, 



D 



0, re Si. 



(4) 



However, the rate depends on X]j. Note that this is possible and even frequent in 
molecular biology, meaning that reactions of the type 5*1 recover the reactant, like in 
the reaction A ^ A + B, with A and B discrete and continuous species, respectively. 
The more complicated case 7^ ^ is treated in section 5. 
The scaled generator has now the form 



AN.f{xc,XD) 



■E 

reSi 



f{xc + ^l?,XD)-.fixc,XD) 



f{xc + J^l?,XD)-f{xc,XD) 



NXrixc) 

NXrixcXo) 



E 

reTZDc\Si 



.n^c + , Xd + 1?) - fixcXo) 



N 



XrixcXo) 



J2 IfixcXo+l^) - fixcXD)] XriXo). 



reTZi 



For / G Cl{E), we may let ^ cxo and obtain the limit generator 



(5) 



AoofiXcXo) 



i J2 K{xchr + E '^rixcXohrj ' V,^ /(xc, Xj,) 



+ E {!{xc,Xo^^^)- !{xc,Xo)\Xr{xc,Xo) 

+ ^ [f{xc,XD + ^^)-f{xc,XD)]Xr{XD). 
r£TZn 

This formal argument indicates that, as N ^ 00, the process converges to a continuous 
PDF (see ([T|)). The state is described by a continuous variable xc and a discrete 
variable Xjy. The discrete variable is a jump process and is piecewise constant. The 
continuous variable evolves according to differential equations depending on X^. It is 
continuous but the vector field describing its evolution changes when Xd jumps. 
This is rigorously justified by the following theorem. 

Theorem 3.1 Let x^ = {x^,Xg) be a jump Markov process as above, starting at 
x^ (0) = {xc{0),X^{0)). Assume that the jump rates Xr, r G TZc U Si and Xr, 
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r € TZdc \ Si are functions of xq G M^^"^ . We define P^g the law of the PDP 
starting at xq — (xc,o, Xd^) whose jump intensities are: 

renol>-Roc\Si 

the transition measure is defined by: 
S^f{z)Q{dz-x) 

^^'^ \reTZoc\Si reTZo 

for X = {xc,Xd), and the vectors fields are given by: 

Assume that Hvvothesis \2.1\ is satisfied and x^ {Q) converges in distribution to xq, 
then x^ converges in distribution to the PDP whose law is Px„ . 

Proof: 

In the following, we work only with scaled variables and simplify the notation by 
omitting the tildes. In other words, we use to denote the rate of all reactions. 

The proof is divided into three steps. We begin our proof by supposing that the 
jump rates and their derivatives with respect to xq are bounded. Hypothesis 12.31 is 
then satisfied. We then prove Theorem 13.1 1 bv a truncation argument. 

Step 1: Tightness for bounded reaction rates. 

We first assume that all rates A,, are bounded as well as their derivatives with 
respect to xc- 

Let x^ be a Markov jump process whose generator is given by An- 
Without loss of generality, we assume that the initial value of the process is deter- 
ministic: a;^(0) = (x^(0),X^(0)) and converges to xq = (a;c,o,^D,o) in M*^^ x 

Let {Yr)reTZ be a sequence of independent standard Poisson proces ses. By Propo- 
sition 1.7, Part 4, and Theorem 4.1, Part 6, of Ethier and Kurtz ( 1986h we know that 
there exists stochastic processes {x'^)Nm in £>(M+;£') such that 

i^it) = x^(0) + ^ 7,y, [ f X,{£^{s))ds] ,t>0. 

ran ^-^0 ^ 

Moreover, for each A'', x^ and x^ have the same distribution. Since we consider only 
the distributions of the processes, we only consider x^ in the following and use the 
same notation for both processes. 

Using the decomposition x^ = {xq,X^), we have 

reTZc ^ ° 

+ E hr^^ fUx^is),XSis))ds 



^ ^^^Y,.(^j\r{xS{s),XS{s))ds 



reTZDc\S 
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and 



re1ZDc\Si 



\r{X^(s))ds. 



We easily prove tightness in £)(R+;R ^^°) of the laws of (Xn ) Afpw by the same proof 
as for Proposition 3.1 in chapter 6 of lEthier and Kurtj ( 19861) and by using the fact 
that the law of Y r is tight in Z?(R+; N), for every r G TZ, according to Theorem 1.4 of 
Billingslevl (|l999h . 

To prove that the laws of (xn) mpn are tight in C '(R~'~ ; R^^*^ ) , we adapt the argument 
of section 2 chapter 11 in lEthier and Kurtj ()l986l) . 

Let Yr{u) = Yr{u) — M be the standard Poisson process centered at its expectation, 
we have: 



J2 j^l?Yr (^Nj^\r{x^is),XS{s))ds 



r£Si 



+ I F{x^{s),X'^{s))ds 



N I 



renDc\Si 



Yr 



Xr{x^{slX'^{s))ds 



N I 



Observe that 



sup — Yr{Nu) — !• 0, a.s. 



N' 



for any A>Q. Since Ar are bounded, it follows that, for all T > 0, 



sup 

te[o,T] 



^^^Yr (n fK{x^{s))ds) + J2 hrYr (n fXrix^is),XSis))d. 



— > 0, a.s. when iV — ^ oo 
Clearly 



sup 

tG[0,T] 



E j7lrYr(fXrix^is),XSis))ds 

-J \ Q V'' 



0, a.s. 



It follows that there exists a random constant going to zero such that, for t, ti, ^2 S 
[0,r], and lli^lloo = sup^gRMcxN^^B 



Xq {t)\ < \x^m + \\F\\oot + KN, 



and 



kc(^i) -2^0(^2) I < \\F\Uti-t2\+2KN, a.s. 



„N 



Tightness of (xn ) nfn in C( 



Jacod and Shirvaev 



987l). chapter 6, section 3b ) . 



) follows by classical criteria (see for instance 



Jacod and Shirvaev! ( 1987h (chapter 6, section 3b), that {.t^} 



We conclude, from 
{{x^,X^)}n is tight in DiR+iE) 

Step 2: Identification of limit points for bounded reaction rates. 



N 
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Let X = {xt)t>o be the canonical process on L)(M+; E), and Pn the law of {x^)t>o 
on this space, for each N G N. 

We know that for each N E N and ip E £. 

tp{xt) - ^p{xq) ~ I ANf{xs)ds 
Jo 

is a P/v-martingale. Equivalently, for each n G N, ti,...,t„ G [0,r], t > r > 0, 
ip G (Cb(^^))" and ^ G f 

Ep^[(ip{xt)-if{xo)- I AN'f{xs)ds]^{xt^,...,Xtj] 

(6) 

Ep„ \ {ip{Xr) - ip{xo) - J ANf{Xs)ds\ Ipixt^,. . . ,XtJ 

Let {PNk)k be a subsequence which converges weakly to a measure P on i?(M+; E'). 
We know that x is P almost surely continuous at every t except for a countable set Dp 
and that for ti,. . . ,i„ outside Dp, Patj^tTj^^ converges weakly to Pt^^^ t„ where 
7rti,....t„ is the projection that carries the point x G I?(R^; E) to the point (xt-^ , ■ • ■ , Xt„) 
ofM"." 

Therefore, it is easy, using dominated convergence theorem and weak convergence 
properties, to let fc — > oo in © and obtain for t, ti, ...,<„, r outside Dp: 

Ep [ [ (p{xt) - ip{xo) - I Aoo'f{xs)ds] ^p{xt-^, ■ . ■ ,xtj] 

(7) 

ip{xr) ~ ifiixa) - J AooV{xs)ds\ -0(a;ti, • ■ • ,a;t„) 

If t G Dp, we choose a sequence (t'') outside Dp such that ^ t with > t. 
Then Pt^^i} converges weakly to Ptt^^ since a; is P-a.s. right continuous in t and Xfk 
converges almost surely to xt- Then, we use ^ with instead of t, let fc — ?> cxd and 
deduce that ([7]) also holds for t G Dp. Similarly, we show that ti, . . . ,tn,r may be 
taken in Dp. 

This shows that the measure P is a solution of the martingale problem associated 
to the generator Aoo on the domain £. 

Hvpothesis 12 .31 enables us to apply Theorem 12 .41 The martingale problem has then 
a unique solution. It follows that the limit P is equal to P^.^ , the law of the PDP, and 
that the whole sequence {Pn)n converges weakly to Pxg. 

Step 3: Conclusion 

Now, we prove Theorem 13.11 with a truncation argument. 
Let e G C°°(M+) such that 

e{x) = l, .TG [0,1], 
e{x) = 0, .T G [2,oo), 

and, for fc > 1 and r E TZ, define 

^fc(^) = ^ (^) ' ^ 6 ^' 

and 

X';{x) = 9kix)Xr{x). 

Then, the problem with Aj; instead of fulfills Hypothesis 12.31 We define x^ = 
(xq I,, f^) the jump Markov process associated to the jump intensities Ajf, starting 
at x^{0). By the preceding result, we know that, for all fc G N, {x^)n£'n converges 
weakly to the PDP Xk in D(R'^;E), whose characteristics are the jump intensities 
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, with corresponding transition measure, and vector fields (obvious definitions as in 
Theorem [3H). 

Then, {{x^)keT^)N(^N converges weakly to (xaO^pm in D(R+\ EY^. 

By Skorohod representation Theorem (see Billingslev ( 197l[ ). Theorem 3.3), up to 
a change of probability space, we may assume that that for all fc S N (a;^ converges 
a.s. to Xk in D{R+;E). 

Let r > and the stopping times 

T*'-=inf{te [0,T], \xkit)\>k}, 

with T*^ = T if {t G [0,r], \xkit)\ > k} = 0. 
Then, for kj eN 

Xk{t) =xi{t), te [0,t' Ar'=], a.s. 

so that r*^ is a.s. non-decreasing. 

Moreover, if x (resp. x^) are the PDP associated to Aoo (resp. the Markov jump 
process associated to An), then 

Xk{t) =xt, te [0,t'=), a.s. 

and if 



'N 



inf{te [0,T], \x^{t)\>k}, 



with T^ = T if{te [0,r], \x^{t)\ > fc} = 0, then 

x^{t)^x^{t), te [0,T^), a.s. 

Let (5 > 0. Observing that if t'^^^ > T — S and dr-six^ ,Xk) < e, where dr-s is 
the distance on D{[0, T ~ 6]; E), then, for enough small e: 

sup \x^ {t)\ < k, a.s. 
te[o,T-5] 

then a.s., > T - 6 a.nd x^ = x^ in [0, T - 5]. Since > t''-^ > T - S; we have 
also, a.s., Xk — X in [0, T — 6] and 

dT-s{x^ ,x) < e. 

We deduce that 

V(5 > 0, P (dr-six^, x)>e) <¥ <T-5)+¥ {dT-s{x^,Xk) > e) . 

Finally, we have 



< 



T-5)=P( sup |a;fe(t)| > fc - 1) < P( sup \x{t)\>k-l). 
te[o,T''] t£[o,T-S] 



By Hypothesis [531 the PDP x can not explode in finite time. Thus for k large, this 
term is small. Then for large N, the second is small. We deduce that x^ converges 
in probability to x in the new probability space. Returning in the original probability 
space, we obtain that x^ converges in distribution to x. 



4 Piecewise deterministic process with jumps 

In this section, we assume that some of the reactions in a subset 52 of TZdc a-re 
such that 7^ is large and scales with N. We set 7^ = jfj^ fo'" r E S2. We define 

5 = SiUS2. 
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The scaled generator has now the form 

1 



reUc 



N~Xr{xc) 



+ E [/(^C + ^r, Xd + Ir) - fixe, Xd)] XrixC, X d) 
rGS2 



fixe + ^^^\Xd)~ fixe, Xd) 



r£TZoc\S 



fi^c + ^1?, Xd + 1?) - fixcXo) 



N\r{xc,XD) 

XrixcXD) 



[fixc,XD+J?)- fixc,XD)]XriXD)- 



(8) 



For / G Cl{E), we may let —> co and obtam the limit generator 

Ao.fiXc,XD) = I Y ~Mxc)lr + E KixcXDhr] ' ^ ccc f ixc , X d) 

+ E [fixC+l?,XD+^^) - fixcXo)] XrixcXD) 

+ E [fixc,XD+J^)- f{xc,XD)]Xrixc,XD) 
r£TZnc\S 

+ Y [fixc,XD + l^)-fixc,XD)]XriXD). 

This formal argument indicates that if — >■ cxd, the process is a piecewise deterministic 
process with jumps both in xc and Xd- In fact, the proof of this can be done easily 
thanks to the result of section 3. 

Indeed, let us introduce the following auxiliary system of reactions involving the 
variable (xj;?^, x^^, X^) G ]R*=^^ x R*^^ x N^^": 



• If r e 7^c, (xc! ,X(j ,XD)^ixc +^lc,Xc ^^d) 



1,N 



1 



2,N vN\ 



If r e S2, (a;c^,a;^^,X^) {x^" ,x^(j" +%,X^ + Yd) 



l.N 2,N ~r yN 



renDc\S2,ixc ,xc ,XD)^ixc +j^lc,Xc ^Xd+Jd) 

ic ^ T) / 2,N vN\ , 1,N 2,N vN , r \ 

• If re Tin, (x^ ,Xc ,Xd)-^(Xc ,Xd+1d) 

The rates of these reactions are Xrix^^ , x'^^ , X^) = Xrix^^ + x"^^ , X^). 

If we choose the initial data {x^{0),0,X^{0)) then this system and the original 
system are equivalent. Indeed, we recover the value of the original system by the 
relation: 

ix^,XS)^ixli''+x'f,XS). 
Conversely, notice that x^ is a sum of a pure jump part and of a continuous one so 

yN\ 2,N yN\ 



l.N 



2,N 



that given (xq , X^ ) , {x(j , Xq , X^ ) can be recovered by isolating from Xq the 
small jumps to obtain x\j^ and the jump of order one (from to obtain a;^^. 
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The auxiliary system corresponds to the fohowing generator: 
1 



E 



f{xh + i,l?,xl,XD)-fix'c,xl,Xn) 



N 



- [f{x^c.^l+l? ,Xd+1?)- f{x}j,xl,XD)\ \r{x}j+xl,XD) 



E 

reSi 



fi^h + J^lr, ^I.Xd) - fixh, xl, Xd) 



■ E f{x\; + ^l?,xl,XD+l?)- f{x\,,xl,XD) 
reHncXS '- 



NXr{xl;+xl,XD) 

Xr{xlj + Xc,Xd) 



(9) 



+ J2 [f{xh,xh,XD+l^)-f{xh,xh,XD)]Xr{XD). 



This generator is of the form as in section 3 but the discrete variable is now {xq, Xu). 
Using the results proved in this section we thus obtain that this auxiliary system of 
reaction converges in distribution to the piecewise continuous deterministic process 
given by the generator 

f{x},,xl,Xo) 

VrG'Rc reSi / 

+ [fi^h^^l + Ir, Xd +1^)^ f{x'c,xl,XD)] Xrixh + 4: ^ d) 

+ Y [fixh,xl,XD+l^)~ f{xl,,xl,XD)]Xrixl;+xl,XD) 
r^TZncXS 

+ Y [f(xh,xl,XD +1^) - f{xl„xl,XD)] XriXn). 

Going back to the original variables, i.e. setting xc = Xq + Xq, we deduce that the 
original system converges in distributions to the piecewise deterministic process with 
generator Aoo- 

To obtain this convergence, we suppose that the jump intensities are C^-difFerentiable 
with respect to the variable and that the limit PDP satisfies Hypothesis 12.11 



5 Averaging 

In this section, we examine the case when ^ is not satisfied by all the discrete vari- 
ables. In this case, the previous results are not valid. We introduce the decomposition 

Xd = iXl„Xl) e X and = {j^'^,-/^'"^) such that 

7f'i-0, re Si. 

This replaces (jj]). The continuous variable xc follows the same characteristics as in 
section [3] 

In the set of the reactions Si, the discrete variable X^ has fast motion, its jumps 
rates are of order N and its jumps are of order 1. 
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More precisely, for r G 5*1 , the state changes according to 

ixc,Xh,Xl) ^ {xc + ^^^,Xh,Xl+-f^'^) 

with the rate N\r{xc,Xjj,X^). 

We now have the following generator for the process : 



ANfixc,X},,Xl) 



reTZc 

E 

reSi 



fi^c + -l?,X],,Xl) - f(xc,X],,Xl) 



fixe + -l?,X}„Xl+j^'^)~ fixe, X}„Xl) 



E 



fi^c + j;^lr,Xh+l^-\xl+j^-^) - fixc,X},,Xl) 



NXrixc) 

N~\rixc,Xl,Xl) 

\rixc,X}j,X'jj) 



+ [fixe, X},+^^-\Xl + j^-^)- fixe, Xh,Xl)]\riXh,X],). 

(10) 

This new model combines slow and fast motions. This leads to double time scale 
evolution which can be further simplified. Contrary to the previous sections, we can 
not take the formal limit of this generator. The idea is to average in fast discrete 
variables X^, and focus on the slow variables ixe,X^), in order to obtain a much 
simpler averaged generator. We assume that X^ takes only a finite number of values: 
there exists a finite set K in N^^"'^ such that X^, e K. 

To this aim, we introduce the following generator depending on ixe,X]j): 

Ac,A- /^(^l,) = E [HXl+j^^^) - hixD] Kixe,X}„Xl). 

reSi 

We assume that for all ixe,X}j), the process associated to the generator A^^ x]-, 
is uniquely ergodic, and we denote by its unique invariant measure. 

Then we define the averaged jump rates: 

MxcX},) = / ~Xrixe,X}„Xl),.,^.^xi,idXl), r 6 S^, 

~Xrixc,Xh)^ f \rixe,X}„Xl)y,^^xl,idXl),renDe\Si, (11) 

MxcXh) = 1 \riXl,Xl)v,^,x],idXl), r e Ud- 

We present now the main result of this section. 

Theorem 5.1 Let x^ ~ ix^ , X^ ^, X^ 2) a jump Markov process with generator 
An, starting at a;^(0) = (a;^(0),X^-^(0),X^2(0))- Assume that the set of values 
of Xjj 2 is finite and that for all ixc,X^), the process associated to the generator 
Ax^ x}^ is uniquely ergodic. Assume also that the jump rate A^, r G T^c nS'i, and Xr, 
r G TZdc \ Si and A^, r G TZdc ^Ti-D, o-fc with respect to xe ■ Define P^^ the law 
of the PDP starting at xq — ixc,o,X^ q) whose jump intensities are: 

Xixe,Xh)= MxcXh), 

re{nDc\Si)ll-Rn 
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with transition measure: 



Ie 



y^C, d) re{TlDc\Si)UTZo 



and vectors fields: 
Assume that 

1. A, Q and F satisfy Hupothesis \2.1\ 

2. {x^{Q),Xl^^{0)) converges in distribution to xq in M^^^ x 

3. there exists Ki > and K2 > such that, for every s G M+, (xc, X]j, X^) G E, 
and for every bounded function g satisfying the centering condition 

[ giXl),.,^,x.idXl) = (12) 



Pf^-'''^g{Xl)<K^e-''^'\ 



9\\ 



where {Ps'^''^")s is the semigroup associated to A^^ x}^- 

if g is a bounded function satisfying the centering condition ^W\j, the Poisson 
equation 



Y [hixc,X}„Xl + ^^■^) - h{xc,X]„Xl)\ ~Xr{xc.X]„Xl) = g{Xl) 

r-eSi 

has a solution given by 

/•CO J 

hixc,Xh,Xl) = - / Pr'^'^giXDds 
Jo 

and this solution is Lipschitz with respect to xc, uniformly in X^. 

Then {x^,X^^) converges in distribution in D(M+;R*^c x N^^o-i) to the POP 
whose law is Pxo ■ 

Remark 5.2 Assumptions 1. and 2. are similar to those made previously. Assump- 
tions 3. and 4. guaranty that it is possible to average with respect to the variable 
X^. Assumption 3. states that the dynamic associated to the generator Ar^^ x\^ 
exponentially mixing, uniformly with respect to {xc,X\)), assumption ^. then follows 
*/-^a;c,x^ depends smoothly on xc- 

Proof 

The steps of the proof are similar to the proof of the Theorem 13.11 We begin 
by supposing that all the jump rates are bounded as well as their derivatives. This 
Hypothesis wiU be removed at the end of the proof. We take E = R^^<^ x 

Step 1: Tightness. 

The proof of the tightness of the laws of {x^ , X^ ) is similar as in section |31 and 
is left to the reader. 

Step 2: Identification of limit points. 

Since we are interested by the evolution of {xc, X^^), it is natural to consider, for 
the generator An, test functions f(xc, X^^) depending only on those variables. Unfor- 
tunately, it remains some terms of order 1 depending on the variable X^ . To overcome 



14 



this prob l em, w e introduce 'perturbe d test functions' (see Papanicolaou et al. ( 1976f ). 



Kushneil (|l994[) . iFouaue et all (|2007l )) 



fN{xc,Xh,Xl) = f{xc,Xh) + j^f\xc,Xh,Xl), 

with f <E £ and / independent of Xj^. We temporary need that / is also with 
bounded second derivatives with respect to xc- The function is chosen such that 
the formal limit of An fN{xci X]^, X'^) does not depend on X^. 
This can be down by taking f-^ such that 

^ [f\xc,Xh,Xl + 7,^-2) - fixe, Xh,Xl)] K{xc, Xh, Xl) = {XD 

(13) 

with 

9xc.x},{Xl) 

'^h)-7rMxc,Xh,Xl)] -V^^fixcXh) 



\reSi 



+ J2 [f(.^c,Xh+l^^')-fixc,X},)]{K{xc,Xh)-Xrixc,Xh,Xl)) 
+ [fixc,Xh+l^^')~f{xc,X},)]{Kixc,X},)~Xr{Xh,Xl)) 

(14) 

Since the jump rates are assumed to be bounded, g^c-f^ clearly bounded. Also, the 
function gxc,x^ is zero mean with respect to the invariant distribution, i.e. g^^^x^ 
satisfy p^ . 

Then, the function satisfying (|13p exists under our assumptions. Moreover, it 
is not difficult to check that is Lipschitz with respect to xc, uniformly with respect 
to X}y. 

Limiting infinitesimal generator. 

We now return to the analysis of the limiting problem. We denote by ^ the law 
of (x^,Xi^_i,Xi^^2),then 

fN{xc{t),Xh{t), Xl{t)) - fN{xc{0), XhiO), XUO)) 



ANfNixcis),X},{s),Xlis))d 



s 







is a Pr TV martingale. Equivalently, for each n 6 N, ti,...,tn G [0,r], t > r > 0, 



Ep,«( [ fN{xt) ~ fN{xa) - I ANfN{xs)ds]'ilj{xt^,...,xt„) 

(15) 

^P^.N { ( fxiXr) ~ fN{xo) - I ANfN{Xs)ds] Ipixti, ■ . ■ ,Xt„) 







Let us now consider a subsequence {Px.Nk)k of {Px,n)n, the laws of {x^,X^^) 
which weakly converges to a measure on _D(M+;]R^^'^ x N*^"'^). 

It is clear that converges to /, uniformly on E. Moreover, the continuity and 
boundedness properties of ensure that An In converges on E towards Aoof, where 
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AoofixcXjj) 

\reTZc reSi J ^ ' 

+ [f{xc,X},+j^^')-f{xc,X},)]K{xc,X},). 

re(nnc\Si}UTlo 

It is the generator of a PDP whose characteristics have been averaged in fast variable 
Equation p3)) gives, as Nk oo: 

^Px ( ( ■^(^*) ^ ■^(^o) ^ / ■Aoofixs)ds ) ■0(2^*1 ,---,Xt„ 



XI 



(17) 

^p^ ( ( /(2;r) - /(a;o) - / ^oo/(a;s)rfs ) V'(a;ti, ■ ■ ■ ,a;t„) 



Recah that we have assumed that / is with respect to xc- It is easy to prove that 
(fT7)l holds for any / S f by choosing a sequence (/n)in £ such that, for all n, is 
with bounded second derivatives and /„ and V/„ are uniformly bounded and converge 
pointwise to / and V/ . 

The measure is then a solution of the martingale problem associated to the 
generator on the domain of the functions in £ independent of X^. 

By the Theorem 12.41 the martingale problem has then a unique solution. We can 
then deduce that {Px,n)n converges weakly to P^g, the law of the PDP defined in the 
Theorem 15.11 

Step 3: Conclusion. 

Since X|, is assumed to live in a finite set, we can remove the boundedness assump- 
tion on the jumps rates as in section 3. We then conclude as in section [31 Theorem 
15. H is proved. 



6 Discontinuous PDP and singular switching 

In this section, our system has two time scales. The switching between fast and slow 
dynamics is governed by the state of a discrete variable. For simplicity, we suppose 
now that there is no other discrete variable and denote by 9, taking values in {0, 1}, 
this unique discrete variable which governs the time scale of the continuous variables. 

The number of molecules belonging to species C are again supposed to be of order 
TV, and we continue to write xc = 

When 6* = 0, the rates of all reactions from TZc U TZdc is of order N . When 6* = 1, 
some reactions (inactive for 6* = 0) become active with much faster rates of order 
where e is a new parameter, supposed to be small. For simplicity we consider that 
there are no TZc type reactions. This would cause only slightly more complicated 
notations. Then for r G TZdc- we introduce the rescaled reaction rates 

Xr{xc,0)^ -^Xr{xc,0) 
Xr{xc,l)= -^A,.(XC,1). 

This system will not have a limit if 9 stays for too long time in the state 1. The 
rate of the reaction changing 9 from state 1 to state is supposed to be of order ^ 
and we also set: 

Xeixc, 1) = eXe{xc, 1)- 
The opposite reaction changing 9 from state to state 1 is written Xg{xc,0)- 
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In these conditions, the generator of the process has the fohowing form: 
AN,ef{xc,0) 



reRoc 



NXrixc, 0) + [f{xc, 1) - f{xc,0)] Xe{xc,0), 



and 

■^N^efixc, 1) 



rGTZoc 



/(^c + ^7.,l)-/(2:c,l) 



N~Xr{xc, 1) + - [f{xc, 0) - fixe, 1)] ~Xeixc, !)■ 



At the hmit (for high N and small e), we will show that the discrete process 9 
inducing kicks in the continuous variable xc is almost surely equal to 0. We are 
interested in the limit distribution of the process xc- 

We introduce the flow (j)i{t,xc), associated to the vector fields 
Fiixc)^ 7,^A,(a:c,l). 

reRnc 

We now state the main result of this section. 
Theorem 6.1 Letx^''' = (x^'^O^'') be a jump Markov process with values 
{0,1}, with generator An, e, starting atx^'^{0) = (^x^''(0), 6'^''=(0)y A ssume that the 
following assumptions hold: 

• a;^''^(0) converges in distribution to xc{0) in ^^'^ as N ex and e — 0. 

• d^''^{0) converges in distribution to as N ^ (X and e ^ 0. 

• The jump rates Xr, for r £ "R-dc, ^ei', 1) (ind Xg{-, 0) are uniformly bounded, 
with respect to xc, and their derivatives are uniformly bounded. Moreover, there 
exists a > 0, such that 



Xe{xc, 1) > a, Wxc G 



Then (x^''^) converges in distribution in LP{[0,T];M.^^'^), for any oo > p > 1, to 
the PDF whose generator is given by 



AooVixc) 



J2 IrMxcO)) • V,c^(a;c) 



\renL 



+Xe{xc,0) / ((^(0i(t,xc))-V5(xc))Ae((^i(t,.Tc),l)e-^'^''(^i(^^^^)^i)'^^dt 
for every ip £ C^R'^^'^). 

Remark 6.2 The PDF associated to the generator Aoo has discontinuities which are 
not present for the process x^''^ . They only appear at the limit e ^ 0. Therefore, the 
convergence does not occur in Z?(R+; [0,T]), indeed creation of discontinuities can not 
occur in L'(M+; [0, T]). As shown below, it is possible to show tightness, then weak 
convergence of the process Xq'"^ in LP{[0,T]). Unfortunately, this topology does not 
imply a uniform bound on the processes and we cannot get rid of the boundedness 
assumptions on the reaction rates as was done in the preceding sections. 

Proof: The proof is divided into four steps. 

Step 1: Limit of 6^^^ 

We begin with the test-function 



fixc,0)^0, /(.Tc,l) = l, 
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Then, AN,ef{xc,()) = Ae(xc, O)lle=o - \>^e{xc, l)lle=i, and 

/■* 1 ~ 

Jo £ 
is a martingale. We then deduce, for every T > and t e [0, T], 



a. 



E / le"."(s)=ic«s <MAr+l (18) 



e 

where Ma is the superior bound of Ae(-,0). Then, O'^-' in L^{n x [0,T]; {0, 1}) 
when e — >■ and N ^ oo. 

Step 2: Tightness of x^'^ 

For aU i G [0,r], we write 



where (Yr)reTioc independent standard Poisson processes. Then 



N.e, 



it) < .T^'^(o)|+ ^ (^7VM(^r + ^''w(s)=idsJJ (19) 

where M is the superior bound of A,.,r G TZdc- Since suptgjoT] jf^r{Nt) is almost 
surely bounded, using ([T5| and (fT^. we have for all > 0, for TV large enough and e 
small enough: 

P(sup |x^^(t)| > ii') < e(A') 

[0,T] 

where e{K) ^ if A' — > oo. 

We introduce BV{0, T) the space of functions of bounded variations on [0, T], with 
its norm, defined for / £ BV{0,T) by 

!l/!lBy(o,r) = ||/||li([o,t])+sup{^ |/(<j+i-/(tj)|, for (tj)^ a finite subdivision of [0,T]} 

Since the processes Yr are non decreasing, we obtain for a subdivision of [0,r], 
denoted by = io < < ■ • ■ < *n = 



i=0 

I „C 



E E ^ f / ^^A,(a;^^(s), 0)]l,.,«(,)=o + -A,.(x'c}^(s), l)le,..^s)=ids) 

-Y:,. (^J^ A^Ar(x^^(s),0)le.,N(s)=o + ^Ar(x^^(s),l)le.,«(^)^idsj 

^ E ^i". ( rA^A,(x^^(s),0)le=,«(,)^o + -A.(xce,A^(s),l)V,«(.)=ids) 
It is then easy to prove that for all ii' > 0, for N large enough and e small enough: 



m4'"\\BV{o.T)>K)<eiK) 



where e{K) ^ if A' — !• oo. 
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The set {/ G BV{0, T), such that ||/||bv(o.t) ^ K} is relatively compact in 
ii([0,T]) (see for instance [Giustil (|l984[ )i. and the set 



{/ G BV{0,T), such that ||/||oo < K and ||/| 



BV{0,T) 



< K} 



is relatively compact in LP{[0,T]) for 1 < p < oo. We then conclude that the family 



of processes {x^''^)^,^^ is tight in L^([0,T]), 1 < p < oo. 



Step 3: Identification of the limit distribution of Xq 
Since we are interested by the limit distribution of x^''^, we introduce test- functions 
depending only on the continuous variable xc G R*^*^ . We then define 

/(a;c,0) = tf[xc), xc S 



where ip ^ 

We want to define /(•, 1) such that 



E 

r&TZoc 



f{xc + j^l?,l)-f{xc,l) 



NXrixc,l)-X0ixc,l)f{xc,l) 



(20) 



-Xe{xc, l)ipixc)- 



Drawing inspiration from the preceding section, we introduce the process yN{-,x) 
starting from x, and whose generator is 



i^{y + j;^ir) - V'(y) 



NXriy,l). 



Since Fi is Lipschitz on M^^^, a result of Kurt d ( 1971 ) states that for all x € 
T > and (5 > 0. 



sup |yAf(s, x) — (f>i{s, x)\ > S 



s<T 



N^oo 



0. 



(21) 



Remember that 0i(-,x) is the fiow associated to the vector field i^i (x) = J^reTtnc '~fr''^r{x, 1) 
and starting at x. 



We also introduce the semigroup [P^j^^^ defined on Bi 



by 



'^tPix) := E {yN{t,x)) g- /o 
It is classical that {Pf^)t>o satisfies the semigroup property and that 

^PfV(a:)=A^PfV(a:)-Ae(x,l)P,^^(x), xSM*^^, e Cb{R''^),t e 
Then we propose: 



fixe, I) 



P 



N 



(Xei-, l)ipj ixc)ds. 



Since Xg{-,1) is bounded below by a > 0, /(•, 1) is well defined. Moreover f{-,l) 
satisfies ([20)) . Indeed, 



An fixe, 1) - ~Xeixc, l)f{xc, 1) 



A 



jvPf {Xg{; 1)^) (xc) ~ ~Xg{xc, l)Pt^ {Xg{; 1)^) i^c) 



dt 



-Pr [Xei;l)ip) {xc)dt 

mt^ooPf (Ae(-,1)^) {xc)-[Xe{;l)p] {xc) 
-{Xe{;l)^) [xc] 
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This test-function / satisfies: 
and 



AN,efixc,0)= 



/(a:c + ^7f,0)-/(xc,0) 



N~Xr{xc,0) 



P. 



XeixcO)- 



Since / S Cb{E), f is in the domain of £, for all N, e. Then, for all < ti, . . 

and V G C6(R*^^^0 



E 



< t 



(22) 



- / AN,ef{Xc'^is),0)tgN.,(^s)^Qds 



'0(xp''(ii),...,x^''(<£)) I = 



We now are searching a possible formulation for the limit generator. For all x S 
and uniformly in t G [0, T], for all T > 0, we have 



P. 



N 



{Xe{-A)v) (x) {M-A)v) {Mt,x))e-fo^^eiMs,. 



Indeed, 



P, 



N 



+E((Ae(-,l)v5) iMt,x))e'^o^^o{v.is.x).i)ds _ (</,i(t, x))e- ^<'(0i(--)aKs) 

= a + 6 

and it can be shown that for all 6 > 0, and t <T, 

\a\ < 5Lxe(-^)v + ^ 1)'^IUIP(™P.'5<T \yN{s,x) - <t)i{s,x)\ > S) 

and 

\b\ < ||A(,(-,l)^||oo (T,5L3^^(. i)+2P(sup,<j.|2/jv(s,a;)-0i(s,a;)| >J 

where L-^^^^ ^-^^ and Lj^^^^^ are the Lipschitz constants of the functions Xe^-, l)ip and 
Ae(-,1). 

By (PT|) and the dominated convergence theorem, we deduce that 



ANf{xc,0) > Aoo^fiixc)- 

N^oo 

The tightness in LP{[0,T])xL^{nx[0,T]; {0, 1}) of the family of processes (; 



and the Skorohod representation theorem imply the existence of a subsequence which 
converges almost surely in i''([0, T]) x ^^(rj x [0, T]; {0, 1}) to {xcit),0)t G LP{[0,T])x 
i^(17x[0,r];{0,l}). This almost surely convergence implies that a new subsequence 
can be extracted such that 



almost surely, and for almost all t G [0,T]. To simplify notations, this subsequence is 



still noted x, 
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Taking N oo and e — > in ([^^ . we deduce by the dominated convergence 
theorem, the boundedness of / and ip, and the hypothesis on a;^''^(0) and 9^''^{0): 



E 



-> E 



ipixcih), . . .,xciti)) 



for almost aU t £ [0, T] and almost all < ti, . . . , < t. We now prove that 



almost surely, and for almost all s G [0, T]. 
First, for s G [0, T], t e M+, we write 



W-s-oce-s-O 



> Aooy^ixcis)) 



(23) 



pN 



E 



Ae(-,1)^) ix^''{s))~ (Xei;l)^) a;c(s))) e" /o ^«WiK-cW).i)<in 

Xei-A)^] (yN{t,x^''{s))) e-/o^<'(?^«(".-c"(-)).i)'^- 



-E 



-E 



A(,(-,l)(p )(0i(i,a;c(.s)))e--''o^<'(y"("'^"''(''))'i)''" 



- (Ae(-,1)(/^) (0i(i,a;c(s)))e-/o^«(^i("^-o(.)),i)rf« 
= ai + 5i 

For all J > 0, Ti > 0, and < < < Ti, 

|ai| < ^ij^^f. i)^ + 2 ||A0(-,l)(y9||ooP(sup„<y^ |?/jv(u,xc''(s)) - <?!'i(u,xc(.s))| > S 
and if t > Ti, 



|ai| <2||A0(-,l)v?||ooe- 



-aTi 



Similarly, for < t < Ti, 

l^^il < l)^lloo (Ti (5L^^(. + 2P (sup„<j,^ - > <5)) 



and if t > Ti , 



< 2||Ae(-,l)</5||ooe- 



-aTi 



We choose Ti sufficiently large and then S small enough and have almost surely, 
for all s e [0,T], t G M+: 



P. 



\ / \ / AT— >oo,e— !-0 



Ar->oo,e-!-0 

By the dominated convergence theorem, we have almost surely, for almost all s G [0, T] , 
(a«(,1V) ix^'^is))dt^ -> 

V / JV-»oo,e— i-O 



Ae(-,1)^) (0i(t,a;c(s)))e-^o^«WiK-c(s)),i)d«rfi, 



Since G C^i(R*^^), we obtain 
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By the dominated convergence theorem, we conclude finaUy that 



(p{xc{t)) - (f{xc{0)) - / Aoo'p{xc{s))ds 



^P{xc{ti),...,xc{te)) =0 



E 

for almost alH, ^i, . . . , ti such that t G [0, T] and < ti . . . < t. 
Step 4-' Uniqueness of the solution of the martingale problem 
By Theorem 12.41 proved in a weaker sense in the Appendix, we conclude that the 

limit xc, more exactly a version of xc, is the PDP whose generator is given by ^oo- 

A Appendix 

Proof of theorem [ 



Let X = {xt)t>o denote the canonical process on _D(M+; E). Recall that we say that 
a probability measure P on _D(]R+; E) is solution of the martingale problem associated 
to the generator A if, for all / G f , 

fixt) - f{xo) - I Af{xs)ds 



Jo 

is a P-martingale. 

Let {Pt)t>o, Px and A denote the semigroup, the probability measure and the 
infinitesimal generator associated to the PDP whose characteristics are F, A, Q, and 
starting from x G E. 

We need the following 

Proposition A.l Assume that Hvvothesis \2.3\ holds, then, for all t G M+, / G Ptf 

is also in £ and 

WPtfWoo < ll/lloo, (24) 

|PJ(yi,i^)-Pt/(2/2,^^)| <c||/||fe^*|2/i-y2|, yi, y2 e M", G (25) 
for some constants c and K depending only on the characteristics of the PDP. 
Proof : Equation ([^ is clear. Let us define, for / G f and -0 G ;Bb(R+ x E) : 

Gfij{t,x) =E,{f{xt)lt<T,+4'{t-Ti,XT,)%>n) 
= f{<P,{t,y),v)H{t,x) 

+ ip{t - s,z)Q{dz](j)^{s,y),v)X{(j)^{s,y),v)H{s,x)ds 

Jo Je 



for {t, x) G X E, with x = (y, v). Then, according to Lemma 27.3 of David ( 1993[ ). 
GJiPit.x) (/(xt)lt<T„ +^(t-T„,XTjWj 

and 

lim G'}^{t,x)=Ptf{x). 
We then deduce by dominated convergence and the strong Markov property: 

Ptfix) = fiMt,y),^)Hit,x) 

+ 11 Pt-sf{z)Q{dz;(j,,{s,y),v)\{Ms,y).i^)H{s,x)ds. (26) 

Jo JE 



That is. P./ : t f-> Pt/ is a fixed point of G 



/• 
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For T > 0, we introduce the Banach space £t = L°°([0, T], f ), with the norm 
llV-llfT = sup e""* (||V^(t,-)||oo +L^(t..)) 

t<E[0,T] 

where a will be fixed hereafter. 

For t G [0,T], [y,u) e E, we set qt[y,v) = f[(j)^{t,y),v)H{t,y,v). Then g : t 
is in Indeed, 

\H{t,y,v)\<l, {y,iy)eE,te[0,T]. 

Also, from the Gronwall Lemma, we have that 0^ is differentiable with respect to y 
and 

\DyMt,y)\<e''\ iy,^)^E, te [o,T]. 

Then we have, for {y, v) ^ E,t ^ [0, T], 

\DyH{t,y,v)\ <f \Dy{\{<t>,{s,y),v)}\ds<Lx I \Dy^.{s.y)\ds < ^ [e'^' - \) . 
Jo Ja ^ 

Hence if a > L 

Ikllf. < (i/ + ^||/||oo)+!|/||oo. 

If G £t, we then easily show that Gf4' G £t- Moreover, if i/ji, ip2 & £t, we have 

\\GfMtr)-GfMtr)\\oo<Mx f \\Mt ~ s, ■) ^ Mt ~ s, ■)\\^ds. (27) 

Jo 

Also, it is easy to see that 

WDyGfMt, ■) - DyGfMt, Olloc <n f e^-lV'i(t - s, •) - V'2(t - s, -^sds 

Jo 

where k is a constant depending only on the characteristics of the PDP. 
Then we easily deduce that 



WGfiJi-GfMs^ <Ki sup^^e-"*! / e''^'-'>ds+ / e'^^'-^'e'^'ds ] - 

\a a — L J 

where ki is a constant depending only on the characteristics of the PDP. Wie now 
choose a sufficiently large and deduce from Picard theorem that G / has a unique fixed 
point in £t- This fixed point is the limit G'^ip for any ?/' G £t- Thus P.f : t H> Ptf is 
that fixed point, and is then in £t- 

The Lipschitz constant Lp^f of Ptf can easily be found with the preceding results 
and Gronwall Lemma. 

□ 



The infinitesimal generator A of Pt is characterized by (cf. for instance lEthier and Kurtz 



(jl986[ ). part 1, chapter 2): 

{fi-A)-^f= / e-f^'Ptfdt, M>0, feC,{E). (28) 
Jo 

Let f e 8, n > K and = (/i - ^)^^/, then (/3 G X'(^) and 
Ptip{x) - (p{x) 



t^o 



> Aip{x), X e E. 



By Proposition lA.H it is easy to see that if Hypothesis 12.31 is satisfied, {jj. — A) ^ 
maps £ into itself if > K. In particular, ip G £ and for all a; G E: 

ip{xt) - <y5(a;) - / ALp{xs)ds (29) 
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is a Pa;-martingale. 

It follows that, for all x e E, and t G IR+, 

Pt(fi(x) - ifi(x) 1 /■* ,,,,,, 
\ ' - - / EA-A^{x,))ds. 

Since Aip{xs) is P^j-a.s. bounded and continuous on [0,ri[. we deduce that 

P.^(x)-^(x) ^ ^^^^ 
t t^o 

It follows that 

A(p{x) = Aip{x), X e E. 

Remark A. 2 In fact, we do not need that (|29p defines a martingale. This can be 
weakened and replaced by the following: 

For all X G E, if € £ and for t CzT, where T is a dense subset o/M^; 

PMx) - ^(.t) = / E, (A^ix,)) ds (30) 
Jo 

In that case, we can define a sequence {tn)n such that i„ € T for all n and tn as 
n — >■ oo. Taking n — >■ cx) m the equality 

Pt (p(x) - (fix) 1 , 

Jq 

we obtain, since Aip{xs) is P^-a.s. bounded and continuous on [0,ri[, 

Aip{x) = Aip{x), X € E. 

We now use a classical argument (see for instance Ethier and Kurt j ( 19861 ). part 
4, chapter 4). Let P^ be another solution of the martingale problem. By the same 
reasoning as before, let f £ £, fi > K and tp = {fi — A)^^f, then 

(p{xt) - (p{x) - / A(p{xs)ds 
Jo 

is a Pr-martingale. 

Let us write ^ 

Exifixt) - / A(p{xs)ds) = (p{x). 
Jo 

Multiply this identity by ^e~^* and integrate on [0, co) to obtain 
(^^ e-^* iMxt) - A^{xt)) dt^ = ^{x). 
Since Aip — A(p, we deduce, from 

(^oo \ poo 

e-^'f{xt)dt = ifix) = / e-^'Ptf{x)dt, pi>K, 



By injectivity of the Laplace transform, this implies 

(fixt)) = Ptf{x),toT almost all t > 0, for all x e E. 

We have proved that the law of a solution of the martingale problem at a fixed time t in 
a dense set o f , is uniquely d etermined. This implies uniqueness for the martingale 
problem (see iBillingslev ( 19991 ) section 14). 

Remark A. 3 Again, the hypothesis that is a solution of the martingale problem 
can be weakened and replaced by the following: 

Ex f{xt) - fix) ~ A(p (xs) ds] =0, 



for x £ E and almost all t G . The previous calculus are similar, since the Lebesgue 
integral is null over a negligeable set. 
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